Repository navigation
Conversation
This was referenced Oct 1, 2026
dnagoda
added this pull request to stack #622
October 2, 2026 16:05
dnagoda
force-pushed
the
dc.compile-once
branch
from
October 2, 2026 16:35
810d2f3 to
8e144a8
Compare
dnagoda
marked this pull request as ready for review
October 2, 2026 16:36
`run` compiled the embedded standard provider (Javy plugin or Shopify Function provider) from its bytes on every call. Even with a warm Wasmtime cache, the Javy providers took about 8 ms per call, so callers that run the same Function many times paid that cost on every run. Keep the compiled providers for the most recent engine in a private cache, and reuse them while callers pass the same engine. A call with a different engine replaces the cache, so it holds one engine at most. The public API does not change.
dnagoda
force-pushed
the
dc.compile-once
branch
from
October 2, 2026 21:35
8e144a8 to
7962a53
Compare
dnagoda
changed the title
The provider cache held its mutex while Module::from_binary compiled a cold provider, so every other lookup waited, including warm cache hits. Look up the provider under the lock, compile it without the lock, and take the lock again to insert it. If the cache moved to another engine during compilation, return the module without caching it, so the cache still holds one engine at most.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Each
engine::runcall compiles the embedded standard provider (Javy plugin or Shopify Function provider) from its bytes. Even with a warm Wasmtime cache, that takes 8 to 12 ms for the Javy providers, so a caller that runs the same Function many times pays it on every run. It is also the main cost when many inputs run in one process (#620).What
runkeeps the compiled providers for the most recentEnginein a private cache. It reuses them while callers pass the same engine (Engine::same).Modulekeeps itsEnginealive, so an unbounded cache would keep every engine a caller ever used.runandFunctionRunParamskeep their signatures, andValidatedModulestays private.Performance
1,000
runcalls with one engine in one process. Release build, Apple M4 Pro, warm Wasmtime compilation cache.mainjs_function_javy_plugin_v3.wasmshopify_functions_javy_v3js_function.wasmjavy_quickjs_provider_v1exit_code.wasmA single CLI run still compiles the provider once. Median of 200 process runs with the Javy plugin v3 fixture: 17.8 ms on
main, 17.0 ms with this PR. The--jsonoutput is identical.Precompiling the providers at build time, as
runtime-enginedoes, would cut that first load to about 1 ms. It would also add about 39 MB to the binary, and it does not change the cost per run once a provider is loaded. It is not part of this PR.Testing
cargo test --locked: 34 unit tests and 23 integration tests pass; one existing test remains ignored.cargo clippy --locked -- -D warningsandcargo fmt --all -- --checkpass.